Evaluation of Image Processing Algorithms

ABSTRACT

One exemplary aspect of this invention pertains to a method to evaluate an image processing algorithm. The method includes varying a parameter of a model of an imaging system and, for each variation of the parameter, calculating with a data processor a corresponding image of a sample; applying an image processing algorithm to the calculated corresponding images of the sample; and determining an ability of the image processing algorithm to detect the variation in the parameter.

TECHNICAL FIELD

The exemplary embodiments of this invention relate generally to a methodof systematically and objectively evaluating and improving theeffectiveness of image processing algorithms, thereby removinguncertainty and variability introduced by human judgment.

BACKGROUND

An image processing algorithm typically calculates a figure of merit fora series of real images, and the image having the largest figure ofmerit is assumed to be the “best” image. Reference in this regard may bemade to, for example, F. C. A. Groen, I. T. Young, G. Lighthart, “Acomparison of different focus functions for use in autofocusalgorithms”, Cytometry, Vol. 6, pgs. 81-91 (1985). The quality andeffectiveness of the algorithm is typically tested by comparing theimage selected using the algorithm with an image that a human selects asthe “best” image. Reference in this regard may be made to, for example,any of the following publications (in addition to the Groen et al.publication noted above): A. Santos, et. al., “Evaluation of autofocusfunctions in molecular cytogenic analysis”, J. Microscopy, Vol 188 (3),pp 264-72, (1997); J. M. Geusebroek, F. Cornelissen, A. Smeulders, H.Geerts, “Robust Autofocusing in Microscopy”, Cytometry, Vol. 39, pgs.1-9 (2000); Y. Sun, S. Duthaler, and B. J. Nelson, “Autofocusing incomputer microscopy—selecting the optimal focus algorithm,” MicroscopyResearch and Technique, Vol. 65, No. 3, pgs. 139-149, 2004; Y. Sun, S.Duthaler, and B. J. Nelson, “Autofocusing algorithm selection incomputer microscopy,” IEEE/RSJ International Conference on IntelligentRobots and Systems (IROS2005), Edmonton, Alberta, Canada, Aug. 2-6,2005; and X. Y. Liu, W. H. Wang, Y. Sun, “Dynamic evaluation ofautofocusing for automated microscopic analysis of blood smear and papsmear”, J. Microscopy Vol. 227(1), pgs. 15-23 (2007).

In a conventional approach, shown in FIG. 1A, a first step applies animage processing algorithm to a series of real images to obtain a figureof merit for each image. A next step then uses a human to rank order theimages by perceived quality. A determination is then made to determineif the rank ordering by human judgment agrees with the order based onthe algorithmic figure of merit. If it does, the result is inconclusivesince both the algorithm and the human judgment may be either correct orincorrect. If the rank ordering based on human judgement is found not toagree with the algorithmic figure of merit the result is alsoinconclusive, since either the algorithm or the human judgment may becorrect.

As may be appreciated, this conventional approach is subjective anderror prone. In addition to the variability of human judgment, for someimaging situations neither the algorithm nor the human actually selectthe “best” image. Thus, conventional methods of evaluating imageprocessing algorithms, as outlined in FIG. 1 A, are inadequate and errorprone.

SUMMARY

The foregoing and other problems are overcome, and other advantages arerealized, in accordance with the exemplary embodiments of thisinvention.

In one exemplary aspect thereof embodiments of this invention provide amethod to evaluate an image processing algorithm. The method includesvarying a parameter of a model of an imaging system and, for eachvariation of the parameter, calculating with a data processor acorresponding image of a sample; applying an image processing algorithmto the calculated corresponding images of the sample; and determining anability of the image processing algorithm to detect the variation in theparameter.

In another exemplary aspect thereof embodiments of this inventionprovide a method to evaluate image compression and decompressionalgorithms. The method comprises, using a data processor, injectingvarying degrees of noise to a calculated image of a sample, producing afirst set of calculated images having varying degrees of imagedegradation; applying at least one noise detecting algorithm to thefirst set of calculated images to evaluate the effectiveness of the atleast one noise detecting algorithm to detect image degradation in thefirst set of calculated images; applying at least two differentcompression/decompression algorithms to the first set of calculatedimages to generate at least second and third sets of calculated images;applying the at least one noise detecting algorithm to the generated atleast second and third sets of calculated images; and determining whichone of the at least two image compression/decompression algorithmsintroduces the least additional image degradation.

In another exemplary aspect thereof embodiments of this inventionprovide a computer-readable storage medium containing computer softwareinstructions, where the execution of the computer software instructionsby a data processor results in operations that comprise varying aparameter of a model of an imaging system and, for each variation of theparameter, calculating a corresponding image of a sample; applying animage processing algorithm to the calculated corresponding images of thesample; and determining an ability of the image processing algorithm todetect the variation in the parameter.

In another exemplary aspect thereof embodiments of this inventionprovide a computer-readable storage medium containing computer softwareinstructions, where the execution of the computer software instructionsby a data processor results in operations that comprise injectingvarying degrees of noise to a calculated image of a sample, producing afirst set of calculated images having varying degrees of imagedegradation; applying at least one noise detecting algorithm to thefirst set of calculated images to evaluate the effectiveness of the atleast one noise detecting algorithm to detect image degradation in thefirst set of calculated images; applying at least two differentcompression/decompression algorithms to the first set of calculatedimages to generate at least second and third sets of calculated images;applying the at least one noise detecting algorithm to the generated atleast second and third sets of calculated images; and determining whichone of the at least two image compression/decompression algorithmsintroduces the least additional image degradation.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other aspects of the embodiments of this invention aremade more evident in the following Detailed Description, when read inconjunction with the attached Drawing Figures, wherein:

FIG. 1A is a logic flow diagram showing a conventional practice forevaluating an image processing algorithm, such as a focusing algorithm.

FIG. 1B is a logic flow diagram showing, in accordance with theexemplary embodiments of this invention, the evaluation of an imageprocessing algorithm using calculated images.

FIG. 2 shows exemplary calculated images of a photomask at three focusvalues.

FIG. 3A shows variance versus focus for calculated images of variousfeatures on a chrome-on-glass photomask, while FIG. 3B shows varianceversus focus for calculated images of various features on an attenuatedphase shift mask.

FIGS. 4A and 4B, collectively referred to as FIG. 4, depict simulatedimage of lines and spaces, where FIG. 4A shows pattern and intensitycontours, and FIG. 4B shows the simulated image.

FIGS. 5A-5C, collectively referred to as FIG. 5, show simulated imagesof lines and spaces of FIG. 4 for a case of a chrome-on-glass mask,where FIG. 5A shows a −280 nm defocus condition, FIG. 5B shows anin-focus condition, and FIG. 5C shows a +280 nm defocus condition.

FIG. 6 shows graph of image standard deviation vs. focus for thechrome-on-glass mask for various line types.

FIGS. 7A and 7B, collectively referred to as FIG. 7, depict simulatedimage of contact holes, where FIG. 7A shows pattern and intensitycontours, and FIG. 7B shows the simulated image.

FIGS. 8A-8C, collectively referred to as FIG. 8, show simulated imagesof contact holes of FIG. 7 for a case of a phase shift mask, where FIG.8A shows a −200 nm defocus condition, FIG. 8B shows an in-focuscondition, and FIG. 8C shows a +200 nm defocus condition.

FIG. 9 shows graph of image standard deviation vs. focus for the phaseshift mask for various line types, and illustrates a condition whereinthe maximum in the standard deviation vs. focus does not occur at 0de-focus.

FIG. 10 is a block diagram of an embodiment of a data processing systemthat is suitable for implementing the exemplary embodiments of thisinvention.

FIG. 11 is a block diagram of an apparatus that is suitable forimplementing the exemplary embodiments of this invention.

FIG. 12 is a logic flow diagram that is descriptive of a method, as wellas operations performed by a computer program product, further inaccordance with the exemplary embodiments of this invention.

DETAILED DESCRIPTION

Image processing algorithms are routinely used to evaluate the qualityof an image or a series of images. Algorithms may be used to evaluate,as non-limiting examples, the focus, illumination uniformity, spatialdistortion, and/or noise of an image, or to detect defects or variationsin spatial extent, in an image relative to a standard. The algorithmprovides a figure of merit intended to select the “best” image from theseries of images, or provide a quantitative measure of variation in theseries of images. Typically, the performance of the algorithm may betested using real images of a sample. Since real images in most, if notall cases contain variations in numerous parameters, and uncertainty inthe value(s) of the parameter of interest, testing of the algorithm iscompromised.

As was noted above, algorithms are often evaluated by comparing the“best” image selected by the algorithm with the image a human wouldselect. Unfortunately, human selection is unreliable, and often does notresult in selection of the best image. Algorithms may also be evaluatedby comparing the results of one algorithm with the results of anotheralgorithm. However, since no “absolute” standard currently exists, onlyrelative comparisons are possible with this approach.

It is assumed that the imaging behavior of any optical system, which maybe referred to without a loss of generality as an imaging system, may beaccurately calculated and modeled and used to replicate the actualimaging behavior of the optical system (see, for example, R. L. Gordon,A. E. Rosenbluth, “Lithographic image simulation for the 21st centurywith 19th century tools”, SPIE, Vol. 5182, pgs. 73-87 (2004)). Inaccordance with the exemplary embodiments of this invention an opticalsystem of interest is modeled, and the model is used to generatecalculated images of a sample. A particular parameter of the opticalsystem of interest, such as focus, is systematically varied in the modelto obtain a series of calculated images at known absolute values of theparticular parameter. An image processing algorithm is applied to theseries of calculated images to determine how well the algorithmdifferentiates the images based on the varied parameter, and howsuccessfully the algorithm selects the “best” image, e.g., the bestfocused image. By the use of this technique the performance of the imageprocessing algorithm can be accurately assessed, as the subjectivity andvariability introduced by the use of a human observer is avoided.

A method in accordance with the exemplary embodiments of this inventionis outlined in FIG. 1B. The method eliminates uncertainty in imageselection by using calculated images in which the “best” image is knowna priori. While described below primarily in the context of a methodused for the specific case of an algorithm to select a best focusedimage, the method is more general and may be applied to image processingalgorithms other than focus algorithms.

By using calculated images rather than real images of a sample, allunwanted variations in the image are eliminated. In addition, by usingcalculated images, the absolute value of the parameter of interest (suchas focus, distortion, noise, linewidth, edge roughness or edge slope) isknown. Thus, accurate evaluation of an image processing algorithmbecomes possible.

FIG. 1B shows an example of a method in accordance with the invention.In FIG. 1B, at Step A, a series of images of a sample are calculated asa function of some parameter (e.g., focus) by modeling the opticalsystem. At Step B an image processing algorithm is applied to the seriesof calculated images to obtain a figure of merit for each image as afunction of the known value in the parameter of interest. At Step C adetermination is made as to whether the image processing algorithmdetects the known variation in the parameter of interest as displayed inthe calculated images. If the result is affirmative, at Step D aconclusive result is declared and the image processing algorithm isassumed to be correct. If the result is negative at Step C a conclusiveresult is also declared, and the image processing algorithm is assumedto be incorrect.

Related to FIG. 1B, an appropriate model of the optical system iscreated. Numerous methods of modeling the performance of an opticalsystem are routinely used, particularly in the field of semiconductorlithography. These methods range from simple optical ray tracing (see,for example, in addition to the above-cited R. L. Gordon, A. E.Rosenbluth, “Lithographic image simulation for the 21st century with19th century tools”, SPIE, Vol. 5182, pgs. 73-87 (2004), also C. Mack,PROLITH: A comprehensive optical lithography model. SPIE OpticalMicrolithography IV, Vol. 538, pgs. 207-220 (1985)), to a full threedimensional solution of Maxwell's equations (see, for example, Z. Zhu,A. Strojwas, “A superfast 3D lithography simulator and its applicationfor ULSI printability analysis”, SPIE Vol. 5377, pgs. 658-669 (2004); K.Toh and A. Neureuther. “Three-dimensional simulation of opticallithography”, In Proceedings: SPIE Vol. 1463, pgs. 356-367 (1991); A.Wong and A. R. Neureuther, “Rigorous three-dimensional time-domainfinite-difference electromagnetic simulation,” IEEE Trans. SemiconductorManufacturing, Vol. 8, No. 4, pgs. 419-431, November 1995; and A. Wong,R. Guerrieri, and A. R. Neureuther, “Massively parallel electromagneticsimulation for photolithographic applications,” IEEE Transactions onCAD, Vol. 14, No. 10, pgs. 1231-1240, October 1995)). In all cases, themodels produce an accurate calculated image of a sample. A parameter ofthe optical system (e.g., focus), or of the sample (e.g., phase) issystematically varied, and a series of images are calculated. The imageprocessing algorithm is then applied to this series of calculatedimages. By correlating the results of the image processing algorithmwith the known variation in the parameter of interest, the ability ofthe algorithm to detect the variation in this parameter can bedetermined unambiguously. Thus, the image processing algorithm can betested to determine how well the algorithm differentiates the imagesbased on the varied parameter, and how successfully the algorithmselects the “best” image, e.g. the best focused image. A specific andnon-limiting example is now described.

Focusing algorithms are often utilized to automatically adjust the focusof an optical microscope to obtain the “sharpest” (i.e., best focused)image. One widely used focus algorithm calculates the variance of animage taken at one focus setting, and compares it to the variance of animage taken at another focus setting. The image with the larger varianceis selected as the best focused image, since a well focused image shouldhave more contrast and hence variance than a poorly focused image.Previously the accuracy of this algorithm could be tested by comparisonwith the best focused images selected by a human (see, for example, X.Y.Liu, W. H. Wang, Y. Sun, “Dynamic evaluation of autofocusing forautomated microscopic analysis of blood smear and pap smear”, J.Microscopy Vol. 227(1), pgs. 15-23 (2007)). Implicit in this approach isthe assumption that the algorithm and/or human will consistently selectthe best focused image. However, the inventors have discovered that thisassumption is not valid.

In the course of developing an autofocusing system for a photomaskrepair tool it was found that operators were unable to reliably producefocused images of a particular class of photomasks. While the operatorswere generally able to reliably focus a chrome-on-glass photomask, theywere unable to consistently focus attenuated phase shift photomasks. Theoperators were simply uncertain as to what constituted a well focusedimage of a phase shift mask due to the complex variation in imagecontrast as the focus was varied. Thus, human judgment was unreliable inthis case.

In accordance with an aspect of this invention the optical system of therepair tool was modeled and images were calculated of bothchrome-on-glass and phase shift photomasks as a function of defocus. Theimages shown in FIG. 2 were calculated for a transmitted light opticalsystem having a 0.9 numerical aperture objective lens, operating at 248nm wavelength, with an illumination coherence of 0.6 sigma. Thechrome-on-glass photomask had regions of 0% and 100% transmission of the248 nm light, while the attenuated phase shift photomask had regions of28% transmission with a phase shift of 168 degrees with respect toregions of 100% transmission, corresponding to measured values for bothtypes of photomasks. Images of various line/space and contact holepatterns were calculated at varying amounts of defocus. The variancealgorithm was then applied to the calculated images, with the resultsplotted in FIG. 3. Each data point in FIG. 3 corresponds to the varianceof a calculated image at a particular defocus, with 0 corresponding toperfect focus.

Each of the seven curves in FIGS. 3A and 3B correspond to a different2-dimensional pattern on the photomask, with FIG. 3A representingresults for the chrome-on-glass photomask, and FIG. 3B representingresults for the attenuated phase shift photomask. The results show thatthe variance algorithm consistently selected the chrome-on-glass imageswith zero defocus (i.e., the maximum variance always occurred for thebest focused images as shown in FIG. 3A), but the algorithm consistentlyselected defocused images of the attenuated phase shift photomasks(i.e., the maximum variance occurred for defocused images as shown inFIG. 3B). The seven curves shown in FIG. 3, representing the differentphotomask patterns employed, show that the sensitivity of the algorithmto defocus, as indicated by the width of the curves, is highly dependenton the specific pattern. The use of the calculated images enabled theaccurate evaluation of the accuracy and precision of the focusalgorithm, and furthermore determination to be made as to whatconstitutes a “best focused” image of both chrome-on-glass andattenuated phase shift photomasks. The use of the calculated imagesfurthermore provides a method to more accurately select the best focusedimage of an attenuated phase shift photomask by adding a predeterminedoffset to the focus selected by the variance algorithm, the offset thatis used being based on the results shown in FIG. 3B.

The method depicted in FIG. 1B may be applied to a wide array of usecases and applications, and is particularly useful in the field ofsemiconductor lithography. For example, detecting defects on a photomaskinvolves complex image processing algorithms. Typically, a photomask isfabricated, imaged in an optical inspection system, and the image iscompared with the design data used to fabricate the photomask. Anydifferences between the image and the design data is attributed todefects. The ability of the inspection tool and algorithm to detectdefects is usually evaluated by building a test photomask withprogrammed defects, and experimentally determining which defects theinspection tool can detect. Using the methods described herein, theoptical system in the inspection tool is modeled to produce calculatedimages of a photomask design. The inspection algorithms are then appliedto the calculated images to determine the ability of the algorithm todetect a defect. Since any type and magnitude of variation in thephotomask can be modeled, the ability of the inspection tool to detectany potential defect can be determined with any degree of precisionrequired. This includes two dimensional and three dimensional spatialerrors in the photomask, as well as variations in transmission, phase,scattered light (as non-limiting examples). In additional, potentialvariations in the inspection tool may also be modeled to determine theirimpact on defect detection, e.g. noise, aberrations and defocus. Thus, acomprehensive evaluation of the inspection tool is made possible, and isnot subject to the uncertainties and variations inherent in a fabricatedtest photomask. Since computations may be performed quickly andinexpensively, a large number of possible defects may be tested fordetectability by the inspection tool. Similarly, by modeling the opticallithography printing tool, the significance of defects detected ormissed by the inspection tool may also be determined.

The foregoing method may be applied in other lithography situations. Forexample, various linewidth, contact area, or line edge roughnessmeasurement algorithms maybe applied to calculated images of a photomaskor semiconductor wafer pattern. Controlled amounts of linewidthvariation, roughness, image noise and image rotation, as non-limitingexamples, can be included in the calculated images, and the impact onthe measurement algorithm evaluated. This procedure may thussignificantly aid in the selection and improvement of measurementalgorithms.

The foregoing method may be applied to any image processing algorithmand to any type of image. For example, landscape or portrait images canbe calculated. Optical systems ranging from, for example, the human eye,to cameras, microscopes, telescopes and binoculars may be modeled.

Imaging systems may include optical light, as well as electron beams orion beams, as examples. Imaging systems may use electromagneticradiation to form an image of a sample, where the electromagneticradiation may be visible light, ultraviolet light, infrared light,x-rays, or microwaves, as non-limiting examples. Blur, lighting,distortion and noise may be varied in each calculated image, and theappropriate image processing algorithm applied to determine howeffective the algorithm is at evaluating the varied parameter (e.g.,blur, lighting, noise).

The exemplary embodiments of this invention may also be used to evaluateimage compression algorithms, and determine how well they preservevarious image qualities such as blur, lighting, distortion and/or noise,after the compression/decompression process. Reference may also be madeto FIG. 12, described in detail below. For example, a single image maybe calculated and transformed by injecting varying degrees of controllednoise, thus producing a set of images (Set A). Various noise detectingalgorithms can be applied to this set of images (Set A) to evaluate theeffectiveness of the algorithms at detecting noise. The Set A of imagesmay then be compressed and decompressed using a compression algorithm,thus forming Set B images. Similarly different compression/decompressionalgorithms can be applied to Set A images to generate Set C, Set D,etc., images. The noise detecting algorithms can then be applied to theSet B,C, D . . . images. The best compression algorithm (i.e., the onethat introduces the least additional noise) can then be determined. Thissame approach can be used to evaluate the effect of various compressionalgorithms on other parameters such as distortion and blur, as twonon-limiting examples.

By example, algorithms to measure noise in an image are described in “Anobjective measure for perceived noise” by Vishwakumara Kayargadde andJean-Bernard Martens, Signal Processing 49 (1996), pgs 187-206, andreferences therein.

Numerous algorithms for compressing and decompressing images (e.g.,jpeg) are also widely used and described in Chapter 27, “The Scientistand Engineer's Guide to Digital Signal Processing” by Steven W. Smith,California Technical Publishing 1997, ISBN 0-9660176-3-3 and in “DigitalImage Compression: Algorithms and Standards” by Weidong Kou, Springer1995, ISBN 978-0792396260.

The exemplary embodiments of this invention thus provide for the use ofsimulated (calculated) images rather than real (actual) images, andprovide precise control of focus variation under known imagingconditions and control of absolute value of focus so as to determinewhich image is in best focus.

As but one non-limiting example, images of typical lithographic patternsare simulated (see, for example, FIG. 4), and a particular set ofoptical conditions are selected. These optical conditions may include,for example, wavelength, numerical aperture, illumination, focus,aberrations and type of sample (e.g., chrome-on-glass photomask, phaseshift photomask). A typical set of patterns, such as lines and spaces,iso lines and spaces and/or contact holes, is selected, and the methodthen generates a series of images at various amounts of defocus. FIG. 5shows an exemplary case of—280 nm defocus, in focus, and +280 nm defocusfor the simulated image of FIG. 4 for a chrome-on-glass mask. A selectedimage processing algorithm is then applied to the series of simulatedimages and a standard deviation of the images is calculated. Adetermination is then made as to whether a best focus is obtained at themaximum standard deviation. FIG. 6 shows an exemplary case of variousline patterns (e.g., E4, E1, etc.) for a chrome-on-glass mask where themaximum in the standard deviation occurs at 0 defocus, and the curvesare symmetrical about the 0 defocus point. Further by example, FIG. 7shows exemplary pattern and image contours, and a simulated image ofcontact holes. FIG. 8 shows an exemplary case of −280 nm defocus, infocus, and +280 nm defocus for the simulated image of FIG. 7 for a phaseshift mask (a MoSi phase shift mask). FIG. 9 shows an exemplary case ofthe various line patterns for the MoSi phase shift mask, where it can benoted the maximum in the standard deviation does not occur at 0 defocus,and the curves are asymmetrical about the 0 defocus point. Thisindicates a “failure” of the image processing (focus) algorithm.

In the FIGS. 4-9 images of chrome-on-glass and phase shift masks weresimulated, and various line space and contact patterns were simulated.The focus was varied for each image type, and a focus algorithm wasapplied to each simulated image. The focus algorithm calculated thevariance of the image, where the best-focused image should correspond tothe image having the largest variance.

The focus algorithm correctly selected the best focused image for thechrome-on-glass mask, but did not select best focused image for thephase shift mask. In fact, the focus algorithm selected an image thatwas approximately 100 nm defocused. However, the focus algorithm maystill be used by offsetting the selected focus for the phase shift maskby a predetermined amount (e.g., 100 nm in this non-limiting example).

FIG. 10 is a block diagram of an exemplary embodiment of a dataprocessing system 10 that is suitable for implementing the exemplaryembodiments of this invention. System 10 includes at least one dataprocessor 12, a user interface 14, such as a graphical user interface(GUI) and a computer-readable storage medium, such as a memory 16, thatstores a plurality of computer software program modules and datastructures 16A-16D. The system 10 may also include a network (NW)interface 18 providing bi-directional connectivity to one or moreexternal local area networks (LANS) and/or wide area networks (WANs),such as the internet.

The memory 16, which may be implemented using any suitable memorytechnology, may include a model of the optical system of interest 16Aand a set of image data representing images 16B calculated using themodel 16A (e.g., calculated images such as those shown in FIGS. 2, 4B,5, 7B and 8). Note that the model 16A may be computed using the system10 using suitable modeling software (which may then also be stored inthe memory 10), or the model 16A may be computed elsewhere andsubsequently sent to the system 10 (e.g., such as through the networkinterface 18) for storage in the memory 16. In another exemplaryembodiment the model of the optical system 16A may be resident atanother computer system, and only the calculated images 16B downloadedto the memory 10. As such, it should be appreciated that the particularembodiment shown in FIG. 10 is merely exemplary of the possibleembodiments that the invention may assume.

The memory 16 may also include one or more image processing algorithms16C, such as a focus algorithm 16D as discussed above, as well as anyadditional computer software that may be used to integrate the operationof the other software and data structures.

During operation the data processor 12 uses and executes the softwareprograms and data structures as described above to objectively evaluateand improve image processing algorithms by applying the algorithms tocalculated images with known properties.

For the purposes of describing and implementing the exemplaryembodiments of this invention reference may also be made to FIG. 11,where there is shown a block diagram of an apparatus 20 that is suitablefor implementing the exemplary embodiments of this invention. Note thatthe blocks shown may be implemented in hardware, as software, or as acombination of hardware and software. In FIG. 11 there is block thatrepresents a mathematical model of the optical system 22. Note that asemployed herein the “model of the optical system” may be a mathematicalmodel or description of the optical system per se, such as a microscope.In addition, the “model of the optical system” may also include amathematical model or description of a sample or samples of interest(e.g., a patterned photomask).

One or more model parameters of interest can be adjusted (represented bythe device 24) to produce a set of calculated images 26 (e.g.,calculated image 1, calculated image 2, . . . , calculated image n),where each calculated image may represent one setting of n settings ofthe device 24 corresponding to one particular value of the parameter orparameters of interest. The parameter or parameters of interest of themodel of the optical system that are varied by the device 24 may be atleast one of focus, lens aberration, lens distortion, illuminationuniformity, illumination noise, detector noise, numerical aperture,wavelength and blur, as non-limiting examples. The parameter orparameters of interest that may be varied by the device 24 may also be asample property, such as at least one of sample feature type, samplefeature size, sample optical properties including at least one oftransmission and phase, reflectivity, transmission and phase shift, asnon-limiting examples.

An image processor 28 embodies at least one type of image processingalgorithm, such as a focus algorithm, and processes the set ofcalculated images 26 to produce a figure of merit (FOM) for each, whichmay then be used to determine how well the image processor 28differentiates the calculated images 26 based on the variedparameter(s), and how successfully the image processor 28 selects the“best” image, e.g., the best focused image, from the set of images 26.One possible, but non-limiting use for the output of the image processor28 is to improve the performance of a variance-based auto-focusingalgorithm, such as one used in an inspection or similar type of tool. Aswas noted above for the exemplary case of the phase shift mask, it ispossible to offset the focus of the imaging system by a predeterminedoffset amount based on the “best” focus position determined by the imageprocessor. It is also possible to adjust an output of the algorithmbased on a result of determining the ability of the image processor todetect the variation in the parameter. These latter operations aredepicted in FIG. 11 by the dashed line representing a feedback path(e.g., offset/adjust) to the image processor 28.

The exemplary embodiments of this invention also pertain to a method,computer program product and an apparatus/system to evaluate imagecompression and decompression algorithms. Referring to the logic flowdiagram of FIG. 12, at Block 12A there is a step/operation of injectingvarying degrees of noise to a calculated image of a sample, producing afirst set of calculated images having varying degrees of imagedegradation. At Block 12B there is a step/operation of applying at leastone noise detecting algorithm to the first set of calculated images toevaluate the effectiveness of the at least one noise detecting algorithmto detect image degradation in the first set of calculated images. AtBlock 12C there is a step/operation of applying at least two differentcompression/decompression algorithms to the first set of calculatedimages to generate at least second and third sets of calculated images.At Block 12D there is a step/operation of applying the at least onenoise detecting algorithm to the generated at least second and thirdsets of calculated images. At Block 12E there is a step/operation ofdetermining which one of the at least two imagecompression/decompression algorithms introduces the least additionalimage degradation. As non-limiting examples, the image degradation maycomprise at least one of blur, illumination non-uniformity, spatialdistortion and lens aberration.

The exemplary embodiments of this invention also pertain to a hardwareplatform or system, which may be referred to as an apparatus, thatincludes at least one data processor coupled with at least one memorythat stores computer program software. Where execution of the softwareby the at least one data processor results in the system evaluating animage processing algorithm by varying a parameter of a model of animaging system and, for each variation of the parameter, calculatingwith the at least one data processor a corresponding image of a sample;applying an image processing algorithm to the calculated correspondingimages of the sample; and determining an ability of the image processingalgorithm to detect the variation in the parameter.

As should be appreciated by one skilled in the art, aspects of thepresent invention may be embodied as a system, method or computerprogram product. Accordingly, aspects of the present invention may takethe form of an entirely hardware embodiment, an entirely softwareembodiment (including firmware, resident software, micro-code, etc.) oran embodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit”, “module” or “system”.Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable software program code embodiedthereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium maybe, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

As such, various modifications and adaptations may become apparent tothose skilled in the relevant arts in view of the foregoing description,when read in conjunction with the accompanying drawings and the appendedclaims. As but some examples, the use of other similar or equivalentoptical system modeling techniques, other than those described in thevarious publications referred to above, may be used by those skilled inthe art. However, all such and similar modifications of the teachings ofthis invention will still fall within the scope of this invention.

Furthermore, some of the features of the examples of this invention maybe used to advantage without the corresponding use of other features. Assuch, the foregoing description should be considered as merelyillustrative of the principles, teachings, examples and exemplaryembodiments of this invention, and not in limitation thereof.

1. A method to evaluate an image processing algorithm, comprising:varying a parameter of a model of an imaging system and, for eachvariation of the parameter, calculating with a data processor acorresponding image of a sample; applying an image processing algorithmto the calculated corresponding images of the sample; and determining anability of the image processing algorithm to detect the variation in theparameter.
 2. The method of claim 1, where the imaging system operatesin accordance with electromagnetic radiation, electrons, or ions to forman image.
 3. The method of claim 1, where the varied parameter comprisesat least one of focus, lens aberration, lens distortion, illuminationuniformity, illumination noise, detector noise, numerical aperture,wavelength and blur.
 4. The method of claim 1, where the variedparameter comprises a sample property.
 5. The method of claim 4, wherethe sample property comprises at least one of sample feature type,sample feature size, sample optical properties including at least one oftransmission and phase, reflectivity, transmission and phase shift. 6.The method of claim 1, where the imaging system comprises a photomaskinspection tool, and where the image processing algorithm is used todetect photomask defects.
 7. The method of claim 1, further comprisingimproving performance of a variance-based auto-focus algorithm byoffsetting the focus by a predetermined amount based on a result of thestep of determining the ability of the image processing algorithm todetect the variation in the parameter.
 8. The method of claim 1, furthercomprising improving the performance of the image processing algorithmby adjusting an output of the algorithm based on a result of the step ofdetermining the ability of the image processing algorithm to detect thevariation in the parameter.
 9. A method to evaluate image compressionand decompression algorithms, comprising: using a data processor,injecting varying degrees of noise to a calculated image of a sample,producing a first set of calculated images having varying degrees ofimage degradation; applying at least one noise detecting algorithm tothe first set of calculated images to evaluate the effectiveness of theat least one noise detecting algorithm to detect image degradation inthe first set of calculated images; applying at least two differentcompression/decompression algorithms to the first set of calculatedimages to generate at least second and third sets of calculated images;applying the at least one noise detecting algorithm to the generated atleast second and third sets of calculated images; and determining whichone of the at least two image compression/decompression algorithmsintroduces the least additional image degradation.
 10. The method ofclaim 9, where the image degradation comprises at least one of blur,illumination non-uniformity, spatial distortion and lens aberration. 11.A computer-readable storage medium containing computer softwareinstructions, the execution of the computer software instructions by adata processor resulting in operations that comprise: varying aparameter of a model of an imaging system and, for each variation of theparameter, calculating a corresponding image of a sample; applying animage processing algorithm to the calculated corresponding images of thesample; and determining an ability of the image processing algorithm todetect the variation in the parameter.
 12. The computer-readable storagemedium of claim 11, where the imaging system operates in accordance withelectromagnetic radiation, electrons, or ions to form an image.
 13. Thecomputer-readable storage medium of claim 11, where the varied parametercomprises at least one of focus, lens aberration, lens distortion,illumination uniformity, illumination noise, detector noise, numericalaperture, wavelength and blur.
 14. The computer-readable storage mediumof claim 11, where the varied parameter comprises a sample property. 15.The computer-readable storage medium of claim 14, where the sampleproperty comprises at least one of sample feature type, sample featuresize, sample optical properties including at least one of transmissionand phase, reflectivity, transmission and phase shift.
 16. Thecomputer-readable storage medium of claim 11, where the imaging systemcomprises a photomask inspection tool, and where the image processingalgorithm is used to detect photomask defects.
 17. The computer-readablestorage medium of claim 11, further comprising an operation of improvingperformance of a variance-based auto-focus algorithm by offsetting thefocus by a predetermined amount based on a result of the step ofdetermining the ability of the image processing algorithm to detect thevariation in the parameter.
 18. The computer-readable storage medium ofclaim 11, further comprising improving the performance of the imageprocessing algorithm by adjusting an output of the algorithm based on aresult of the step of determining the ability of the image processingalgorithm to detect the variation in the parameter.
 19. Acomputer-readable storage medium containing computer softwareinstructions, the execution of the computer software instructions by adata processor resulting in operations that comprise: injecting varyingdegrees of noise to a calculated image of a sample, producing a firstset of calculated images having varying degrees of image degradation;applying at least one noise detecting algorithm to the first set ofcalculated images to evaluate the effectiveness of the at least onenoise detecting algorithm to detect image degradation in the first setof calculated images; applying at least two differentcompression/decompression algorithms to the first set of calculatedimages to generate at least second and third sets of calculated images;applying the at least one noise detecting algorithm to the generated atleast second and third sets of calculated images; and determining whichone of the at least two image compression/decompression algorithmsintroduces the least additional image degradation.
 20. Thecomputer-readable storage medium of claim 19, where the imagedegradation comprises at least one of blur, illumination non-uniformity,spatial distortion and lens aberration.